Performance analysis of robustness of BERT model under attack

Abstract With the aim of testing robustness machine learning models, this paper tests performance five classification models based on IMDB datasets. Furthermore, in work, two types sentence embeddings generated by word2vec and BERT are added with noise normal distribution different intensities. They fed into Support Vector Machine for testing. The experimental results show that model slowly decreases as intensity is increased. BERT-based sentiment embedding reduces less than Word2vec-based embedding. This believes phenomenon can be used a basis to support representation learning. At same time, also noted previous work experimented perturbating replacing words original text dataset, causing drop sharply. However, experiment tested tampering, were stable. In paper, it assumed word substitution interference possible situations when operating model. Therefore, phenomena, plans design more detailed experiments next check